Non-parametric foreground subtraction for 21cm epoch of reionization experiments

One of the problems facing experiments designed to detect redshifted 21cm emission from the epoch of reionization (EoR) is the presence of foregrounds which exceed the cosmological signal in intensity by orders of magnitude. When fitting them so that they can be removed, we must be careful to minimize ‘overfitting’, in which we fit away some of the cosmological signal, and ‘underfitting’, in which real features of the foregrounds cannot be captured by the fit, polluting the signal reconstruction. We argue that in principle it would be better to fit the foregrounds non-parametrically – allowing the data to determine their shape – rather than selecting some functional form in advance and then fitting its parameters. Non-parametric fits often suffer from other problems, however. We discuss these before suggesting a nonparametric method, Wp smoothing, which seems to avoid some of them. After outlining the principles of Wp smoothing we describe an algorithm used to implement it. Some useful results for implementing an alternative algorithm are given in an appendix. We apply Wp smoothing to a synthetic data cube for the Low Frequency Array (LOFAR) EoR experiment. This cube includes realistic models for the signal, foregrounds, instrumental response and noise. The performance of Wp smoothing, measured by the extent to which it is able to recover the variance of the cosmological signal and to which it avoids the fitting residuals being polluted by leakage of power from the foregrounds, is compared to that of a parametric fit, and to another non-parametric method (smoothing splines). We find that Wp smoothing is superior to smoothing splines for our application, and is competitive with parametric methods even though in the latter case we may choose the functional form of the fit with advance knowledge of the simulated foregrounds. Finally, we discuss how the quality of the fit is affected by the frequency resolution and range, by the characteristics of the cosmological signal and by edge effects.